09. Understanding the Training Process

Creating a DQN Reinforcement Learning Trading Agent

Overview of Learning Components

• Financial Data Pre-Processing
  • Feature selection and technical indicators
  • Rolling normalization
• State and Action Spaces
  • State: Features dataset post-preprocessing
  • Action: Possible trade decisions
• Agent Essentials
  • DQN reward function
  • Act function
  • Experience replay function

Training Process Step-by-Step

1. Data Preparation:

   • Collect financial data
   • Define and normalize features
   • Split into training and test sets
   • Optional validation set for overfitting assessment

2. Environment Setup:

   • Define state and action spaces

3. Coding the Agent:

   • Start training in episodes: Each episode runs through the full dataset

4. Iterative Actions:

   • Get initial state at t=0, perform action, receive next state t=1
   • Calculate reward, save experience, and update to t=t+1

5. Experience Replay:

   • Runs after the initial n steps
   • Samples a mini-batch for model fitting
   • Decays epsilon in the act function
   • Continues iterating

Next Steps

• Upcoming lesson will address practical implementation considerations.

Supplementary Material - Validation in Training

The following Medium article gives an overview of using a validation set while training. We are not doing this in this course, but it is generally regarded as best practice to do this.

About Train, Validation and Test Sets in Machine Learning